7 lessons from the early days of generative AI

Like many great love affairs, generative artificial intelligence launched with high intensity and a sense of inevitability. Now that the honeymoon phase is over, enterprises are seeking to mature the relationship from simple pilots and experimentation to high-value implementations that have measurable and sustainable impact.

There are several lessons to learn from early implementations of generative AI, according to Aamer Baig, a senior partner at McKinsey & Company and global co-leader of McKinsey Technology. Speaking at the recent MIT Sloan CIO Symposium, Baig said generative AI presents an opportunity to transform and reimagine business and technology functions in ways that weren’t possible before. At the same time, key issues need to be resolved for it to deliver value beyond early chatbot and content-generation use cases.

This inflection point is reflected in AI adoption numbers. In a 2024 McKinsey global survey on AI, 65% of respondents said that their organizations were regularly using generative AI in at least one business function, up from one-third last year. Yet while single projects are underway in full force, McKinsey found that only 10% of responding companies had successfully implemented generative AI at scale for any use case. 

That disconnect underscores the need for CIOs and digital leaders to spearhead strategies to turn generative AI’s promise into tangible business value. Harnessing this transformational power will require organizations to rewire how they work and put muscle into facilitating change. With that in mind, here are seven “hard truths” Baig said companies must learn for wider AI implementation. 

1. Not all use cases are equal. 

Given how easy it is to get started with generative AI, organizations have scads of initiatives underway — including many that aren’t on the radar of IT and top management. A lot of efforts are scattershot and don’t contribute to the bottom line, Baig said. In the McKinsey AI survey, only 15% of companies reported an earnings improvement from generative AI initiatives.

To find the signal in the noise, Baig recommends plotting generative AI on a value-versus-feasibility matrix to find the use cases that are most worth investment. “One of the most important roles a CIO can play is getting the organization to focus on initiatives that drive real business value, are technologically feasible, and have manageable risk,” Baig said.

2. It’s not just about models but the entire tech stack.

Scaling generative AI is about more than models. Even the simplest uses require about 20 to 30 elements, including large language models, data, gateways, prompt engineering, security, and more. The focus should be on assembling — and, more importantly, integrating — the entire technology stack. “It’s about putting a jigsaw puzzle together. The sum of the parts should be greater than each part individually,” Baig said.

Automation and orchestration are also important components of the broader generative AI ecosystem. While companies have successfully automated portions of the workflow, end-to-end automation is critical for enterprise-scale generative AI, Baig said.

Leading the AI-Driven Organization

In person at MIT Sloan

Register Now

3. Manage costs before they manage you. 

The long-standing orthodoxies governing cost estimation for technology programs don’t translate to generative AI implementations. While it’s relatively easy and inexpensive to get up and running with generative AI, the lower upfront costs don’t accurately reflect the overall economics, particularly when it comes to change management. 

Traditionally, organizations have had comparable budgets for change management and technology. With generative AI, change management can cost up to three times the price tag of the technology itself, Baig said. This is due to the need to overhaul business processes, workflows, key performance indicators, and policies along with newer elements such as prompt engineering and intellectual property control. 

Budget allocation for system maintenance is also different for generative AI implementations. While traditional budget allocations for maintenance are 15% to 30% of deployment costs, the expense for generative AI system maintenance can be as much as what was spent on system development. Making the right or wrong decision on platform choice or scale of automation can also have a dramatic effect on cost. “Don’t get [fooled] by a very easy upfront implementation case, because the overall business case is much more complicated,” Baig said. “Big decisions upfront really, really matter.”

4. Tame the proliferation of tools and technology. 

There are simply too many tools at play with generative AI, mirroring the deployment path organizations followed as they moved to the cloud and software-as-a-service. Baig advised that organizations figure out where standardization is possible, with an emphasis on teams’ productivity.

5. Assemble teams. 

Organizations should focus on building teams with an eye toward delivering value. Often, generative AI and AI efforts remain relegated to skunkworks initiatives driven by a few talented people. To get it right, organizations need to structure work around product-oriented pods and integrated teams, with a commitment to building platforms. Visibility from top management is just as important, to ensure that product and platform teams are organized, focused, and working at pace, Baig said. 

6. Get the right data, not perfect data. 

Data is a daunting challenge for most organizations, and it can be an impediment to generative AI projects. Focusing on data domains that can be applied to multiple use cases is a good way to address the problem and get started. “That usually ends up being three or four domains that can be applied to high-priority business challenges … resulting in delivery of something that actually gets to production and scale,” Baig said.

7. Reuse it or lose it. 

There is so much happening with generative AI that formulating reuse strategies for models, prompts, data, and use cases is crucial to accelerating time to delivery, keeping business users happy, and, ultimately, delivering sustainable impact.

The payoff for addressing these seven hard truths is unparalleled growth and innovation opportunities. “Our research says the value of generative AI is as much as $4.4 trillion in economic impact,” Baig said. “There’s value to doing it well, doing it quickly, but making sure you’re doing it safely and at scale.”

Read next: Want to make the most of generative AI? Use your imagination